

<!DOCTYPE html>
<html class="writer-html5" lang="zh" >
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>蜘蛛合约 &mdash; Scrapy 2.3.0 文档</title>
  

  
  <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
  <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster.custom.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster.bundle.min.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster-sideTip-shadow.min.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster-sideTip-punk.min.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster-sideTip-noir.min.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster-sideTip-light.min.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/tooltipster-sideTip-borderless.min.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/micromodal.css" type="text/css" />
  <link rel="stylesheet" href="../_static/css/sphinx_rtd_theme.css" type="text/css" />

  
  
  
  

  
  <!--[if lt IE 9]>
    <script src="../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
        <script src="../_static/jquery.js"></script>
        <script src="../_static/underscore.js"></script>
        <script src="../_static/doctools.js"></script>
        <script src="../_static/language_data.js"></script>
        <script src="../_static/js/hoverxref.js"></script>
        <script src="../_static/js/tooltipster.bundle.min.js"></script>
        <script src="../_static/js/micromodal.min.js"></script>
    
    <script type="text/javascript" src="../_static/js/theme.js"></script>

    
    <link rel="index" title="索引" href="../genindex.html" />
    <link rel="search" title="搜索" href="../search.html" />
    <link rel="next" title="常用做法" href="practices.html" />
    <link rel="prev" title="调试spiders" href="debug.html" /> 

<script data-ad-client="ca-pub-1466963416408457" async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>


<body class="wy-body-for-nav">

   
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >
          

          
            <a href="../index.html" class="icon icon-home" alt="Documentation Home"> Scrapy
          

          
          </a>

          
            
            
              <div class="version">
                2.3
              </div>
            
          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <p class="caption"><span class="caption-text">第一步</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../intro/overview.html">Scrapy一目了然</a></li>
<li class="toctree-l1"><a class="reference internal" href="../intro/install.html">安装指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../intro/tutorial.html">Scrapy 教程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../intro/examples.html">实例</a></li>
</ul>
<p class="caption"><span class="caption-text">基本概念</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="commands.html">命令行工具</a></li>
<li class="toctree-l1"><a class="reference internal" href="spiders.html">蜘蛛</a></li>
<li class="toctree-l1"><a class="reference internal" href="selectors.html">选择器</a></li>
<li class="toctree-l1"><a class="reference internal" href="items.html">项目</a></li>
<li class="toctree-l1"><a class="reference internal" href="loaders.html">项目加载器</a></li>
<li class="toctree-l1"><a class="reference internal" href="shell.html">Scrapy shell</a></li>
<li class="toctree-l1"><a class="reference internal" href="item-pipeline.html">项目管道</a></li>
<li class="toctree-l1"><a class="reference internal" href="feed-exports.html">Feed 导出</a></li>
<li class="toctree-l1"><a class="reference internal" href="request-response.html">请求和响应</a></li>
<li class="toctree-l1"><a class="reference internal" href="link-extractors.html">链接提取器</a></li>
<li class="toctree-l1"><a class="reference internal" href="settings.html">设置</a></li>
<li class="toctree-l1"><a class="reference internal" href="exceptions.html">例外情况</a></li>
</ul>
<p class="caption"><span class="caption-text">内置服务</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="logging.html">登录</a></li>
<li class="toctree-l1"><a class="reference internal" href="stats.html">统计数据集合</a></li>
<li class="toctree-l1"><a class="reference internal" href="email.html">发送电子邮件</a></li>
<li class="toctree-l1"><a class="reference internal" href="telnetconsole.html">远程登录控制台</a></li>
<li class="toctree-l1"><a class="reference internal" href="webservice.html">Web服务</a></li>
</ul>
<p class="caption"><span class="caption-text">解决具体问题</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../faq.html">常见问题</a></li>
<li class="toctree-l1"><a class="reference internal" href="debug.html">调试spiders</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">蜘蛛合约</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#custom-contracts">定制合同</a></li>
<li class="toctree-l2"><a class="reference internal" href="#detecting-check-runs">正在检测检查运行</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="practices.html">常用做法</a></li>
<li class="toctree-l1"><a class="reference internal" href="broad-crawls.html">宽爬行</a></li>
<li class="toctree-l1"><a class="reference internal" href="developer-tools.html">使用浏览器的开发人员工具进行抓取</a></li>
<li class="toctree-l1"><a class="reference internal" href="dynamic-content.html">选择动态加载的内容</a></li>
<li class="toctree-l1"><a class="reference internal" href="leaks.html">调试内存泄漏</a></li>
<li class="toctree-l1"><a class="reference internal" href="media-pipeline.html">下载和处理文件和图像</a></li>
<li class="toctree-l1"><a class="reference internal" href="deploy.html">部署蜘蛛</a></li>
<li class="toctree-l1"><a class="reference internal" href="autothrottle.html">AutoThrottle 扩展</a></li>
<li class="toctree-l1"><a class="reference internal" href="benchmarking.html">标杆管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="jobs.html">作业：暂停和恢复爬行</a></li>
<li class="toctree-l1"><a class="reference internal" href="coroutines.html">协同程序</a></li>
<li class="toctree-l1"><a class="reference internal" href="asyncio.html">asyncio</a></li>
</ul>
<p class="caption"><span class="caption-text">扩展Scrapy</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="architecture.html">体系结构概述</a></li>
<li class="toctree-l1"><a class="reference internal" href="downloader-middleware.html">下载器中间件</a></li>
<li class="toctree-l1"><a class="reference internal" href="spider-middleware.html">蜘蛛中间件</a></li>
<li class="toctree-l1"><a class="reference internal" href="extensions.html">扩展</a></li>
<li class="toctree-l1"><a class="reference internal" href="api.html">核心API</a></li>
<li class="toctree-l1"><a class="reference internal" href="signals.html">信号</a></li>
<li class="toctree-l1"><a class="reference internal" href="exporters.html">条目导出器</a></li>
</ul>
<p class="caption"><span class="caption-text">其余所有</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../news.html">发行说明</a></li>
<li class="toctree-l1"><a class="reference internal" href="../contributing.html">为 Scrapy 贡献</a></li>
<li class="toctree-l1"><a class="reference internal" href="../versioning.html">版本控制和API稳定性</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../index.html">Scrapy</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../index.html" class="icon icon-home"></a> &raquo;</li>
        
      <li>蜘蛛合约</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
            
        
      </li>
    
  </ul>

  
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="spiders-contracts">
<span id="topics-contracts"></span><h1>蜘蛛合约<a class="headerlink" href="#spiders-contracts" title="永久链接至标题">¶</a></h1>
<p>测试蜘蛛会变得特别烦人，虽然没有什么可以阻止你编写单元测试，但是任务会很快变得很麻烦。Scrapy提供了一种综合的方法，可以通过合同的方式测试你的蜘蛛。</p>
<p>这允许您通过硬编码一个示例URL来测试蜘蛛的每个回调，并检查回调如何处理响应的各种约束。每个合同的前缀都是 <code class="docutils literal notranslate"><span class="pre">&#64;</span></code> 并包含在docstring中。请参见以下示例：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">response</span><span class="p">):</span>
    <span class="sd">&quot;&quot;&quot; This function parses a sample response. Some contracts are mingled</span>
<span class="sd">    with this docstring.</span>

<span class="sd">    @url http://www.amazon.com/s?field-keywords=selfish+gene</span>
<span class="sd">    @returns items 1 16</span>
<span class="sd">    @returns requests 0 0</span>
<span class="sd">    @scrapes Title Author Year Price</span>
<span class="sd">    &quot;&quot;&quot;</span>
</pre></div>
</div>
<p>此回调使用三个内置合同进行测试：</p>
<span class="target" id="module-scrapy.contracts.default"></span><dl class="py class">
<dt id="scrapy.contracts.default.UrlContract">
<em class="property">class </em><code class="sig-prename descclassname">scrapy.contracts.default.</code><code class="sig-name descname">UrlContract</code><a class="headerlink" href="#scrapy.contracts.default.UrlContract" title="永久链接至目标">¶</a></dt>
<dd><p>本合同 (<code class="docutils literal notranslate"><span class="pre">&#64;url</span></code> )设置检查此蜘蛛的其他合同条件时使用的示例URL。本合同是强制性的。运行检查时，忽略所有缺少此协定的回调：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@url</span> <span class="n">url</span>
</pre></div>
</div>
</dd></dl>

<dl class="py class">
<dt id="scrapy.contracts.default.CallbackKeywordArgumentsContract">
<em class="property">class </em><code class="sig-prename descclassname">scrapy.contracts.default.</code><code class="sig-name descname">CallbackKeywordArgumentsContract</code><a class="headerlink" href="#scrapy.contracts.default.CallbackKeywordArgumentsContract" title="永久链接至目标">¶</a></dt>
<dd><p>本合同 (<code class="docutils literal notranslate"><span class="pre">&#64;cb_kwargs</span></code> )设置 <a class="reference internal" href="request-response.html#scrapy.http.Request.cb_kwargs" title="scrapy.http.Request.cb_kwargs"><code class="xref py py-attr docutils literal notranslate"><span class="pre">cb_kwargs</span></code></a> 示例请求的属性。它必须是有效的JSON字典。:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@cb_kwargs</span> <span class="p">{</span><span class="s2">&quot;arg1&quot;</span><span class="p">:</span> <span class="s2">&quot;value1&quot;</span><span class="p">,</span> <span class="s2">&quot;arg2&quot;</span><span class="p">:</span> <span class="s2">&quot;value2&quot;</span><span class="p">,</span> <span class="o">...</span><span class="p">}</span>
</pre></div>
</div>
</dd></dl>

<dl class="py class">
<dt id="scrapy.contracts.default.ReturnsContract">
<em class="property">class </em><code class="sig-prename descclassname">scrapy.contracts.default.</code><code class="sig-name descname">ReturnsContract</code><a class="headerlink" href="#scrapy.contracts.default.ReturnsContract" title="永久链接至目标">¶</a></dt>
<dd><p>本合同 (<code class="docutils literal notranslate"><span class="pre">&#64;returns</span></code> ）为蜘蛛返回的项和请求设置下限和上限。上限是可选的：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@returns</span> <span class="n">item</span><span class="p">(</span><span class="n">s</span><span class="p">)</span><span class="o">|</span><span class="n">request</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="p">[</span><span class="nb">min</span> <span class="p">[</span><span class="nb">max</span><span class="p">]]</span>
</pre></div>
</div>
</dd></dl>

<dl class="py class">
<dt id="scrapy.contracts.default.ScrapesContract">
<em class="property">class </em><code class="sig-prename descclassname">scrapy.contracts.default.</code><code class="sig-name descname">ScrapesContract</code><a class="headerlink" href="#scrapy.contracts.default.ScrapesContract" title="永久链接至目标">¶</a></dt>
<dd><p>本合同 (<code class="docutils literal notranslate"><span class="pre">&#64;scrapes</span></code> ）检查回调返回的所有项是否具有指定的字段：：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@scrapes</span> <span class="n">field_1</span> <span class="n">field_2</span> <span class="o">...</span>
</pre></div>
</div>
</dd></dl>

<p>使用 <a class="reference internal" href="commands.html#std-command-check"><code class="xref std std-command docutils literal notranslate"><span class="pre">check</span></code></a> 运行合同检查的命令。</p>
<div class="section" id="custom-contracts">
<h2>定制合同<a class="headerlink" href="#custom-contracts" title="永久链接至标题">¶</a></h2>
<p>如果您发现您需要比内置Scrapy契约更多的功能，那么可以使用 <a class="reference internal" href="settings.html#std-setting-SPIDER_CONTRACTS"><code class="xref std std-setting docutils literal notranslate"><span class="pre">SPIDER_CONTRACTS</span></code></a> 设置：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">SPIDER_CONTRACTS</span> <span class="o">=</span> <span class="p">{</span>
    <span class="s1">&#39;myproject.contracts.ResponseCheck&#39;</span><span class="p">:</span> <span class="mi">10</span><span class="p">,</span>
    <span class="s1">&#39;myproject.contracts.ItemValidate&#39;</span><span class="p">:</span> <span class="mi">10</span><span class="p">,</span>
<span class="p">}</span>
</pre></div>
</div>
<p>每个合同必须继承自 <a class="reference internal" href="#scrapy.contracts.Contract" title="scrapy.contracts.Contract"><code class="xref py py-class docutils literal notranslate"><span class="pre">Contract</span></code></a> 可以覆盖三种方法：</p>
<span class="target" id="module-scrapy.contracts"></span><dl class="py class">
<dt id="scrapy.contracts.Contract">
<em class="property">class </em><code class="sig-prename descclassname">scrapy.contracts.</code><code class="sig-name descname">Contract</code><span class="sig-paren">(</span><em class="sig-param"><span class="n">method</span></em>, <em class="sig-param"><span class="o">*</span><span class="n">args</span></em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/scrapy/contracts.html#Contract"><span class="viewcode-link">[源代码]</span></a><a class="headerlink" href="#scrapy.contracts.Contract" title="永久链接至目标">¶</a></dt>
<dd><dl class="field-list simple">
<dt class="field-odd">参数</dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>method</strong> (<a class="reference external" href="https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable" title="(在 Python v3.9)"><em>collections.abc.Callable</em></a>) -- 与合同关联的回调函数</p></li>
<li><p><strong>args</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(在 Python v3.9)"><em>list</em></a>) -- 传入docstring的参数列表（空格分隔）</p></li>
</ul>
</dd>
</dl>
<dl class="py method">
<dt id="scrapy.contracts.Contract.adjust_request_args">
<code class="sig-name descname">adjust_request_args</code><span class="sig-paren">(</span><em class="sig-param"><span class="n">args</span></em><span class="sig-paren">)</span><a class="reference internal" href="../_modules/scrapy/contracts.html#Contract.adjust_request_args"><span class="viewcode-link">[源代码]</span></a><a class="headerlink" href="#scrapy.contracts.Contract.adjust_request_args" title="永久链接至目标">¶</a></dt>
<dd><p>这接收了 <code class="docutils literal notranslate"><span class="pre">dict</span></code> 作为包含请求对象的默认参数的参数。 <a class="reference internal" href="request-response.html#scrapy.http.Request" title="scrapy.http.Request"><code class="xref py py-class docutils literal notranslate"><span class="pre">Request</span></code></a> 默认情况下使用，但可以使用 <code class="docutils literal notranslate"><span class="pre">request_cls</span></code> 属性。如果链中的多个合同定义了此属性，则使用最后一个。</p>
<p>必须返回相同或修改过的版本。</p>
</dd></dl>

<dl class="py method">
<dt id="scrapy.contracts.Contract.pre_process">
<code class="sig-name descname">pre_process</code><span class="sig-paren">(</span><em class="sig-param"><span class="n">response</span></em><span class="sig-paren">)</span><a class="headerlink" href="#scrapy.contracts.Contract.pre_process" title="永久链接至目标">¶</a></dt>
<dd><p>这允许在将样本请求传递到回调之前，对从该请求接收的响应进行各种检查。</p>
</dd></dl>

<dl class="py method">
<dt id="scrapy.contracts.Contract.post_process">
<code class="sig-name descname">post_process</code><span class="sig-paren">(</span><em class="sig-param"><span class="n">output</span></em><span class="sig-paren">)</span><a class="headerlink" href="#scrapy.contracts.Contract.post_process" title="永久链接至目标">¶</a></dt>
<dd><p>这允许处理回调的输出。迭代器在传递给这个钩子之前被转换为列表化的。</p>
</dd></dl>

</dd></dl>

<p>提高 <a class="reference internal" href="#scrapy.exceptions.ContractFail" title="scrapy.exceptions.ContractFail"><code class="xref py py-class docutils literal notranslate"><span class="pre">ContractFail</span></code></a> 从 <a class="reference internal" href="#scrapy.contracts.Contract.pre_process" title="scrapy.contracts.Contract.pre_process"><code class="xref py py-class docutils literal notranslate"><span class="pre">pre_process</span></code></a> 或 <a class="reference internal" href="#scrapy.contracts.Contract.post_process" title="scrapy.contracts.Contract.post_process"><code class="xref py py-class docutils literal notranslate"><span class="pre">post_process</span></code></a> 如果未达到预期：</p>
<dl class="py class">
<dt id="scrapy.exceptions.ContractFail">
<em class="property">class </em><code class="sig-prename descclassname">scrapy.exceptions.</code><code class="sig-name descname">ContractFail</code><a class="reference internal" href="../_modules/scrapy/exceptions.html#ContractFail"><span class="viewcode-link">[源代码]</span></a><a class="headerlink" href="#scrapy.exceptions.ContractFail" title="永久链接至目标">¶</a></dt>
<dd><p>合同失败时出现的错误</p>
</dd></dl>

<p>下面是一个演示合同，它检查收到的响应中是否存在自定义头：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">scrapy.contracts</span> <span class="kn">import</span> <span class="n">Contract</span>
<span class="kn">from</span> <span class="nn">scrapy.exceptions</span> <span class="kn">import</span> <span class="n">ContractFail</span>

<span class="k">class</span> <span class="nc">HasHeaderContract</span><span class="p">(</span><span class="n">Contract</span><span class="p">):</span>
    <span class="sd">&quot;&quot;&quot; Demo contract which checks the presence of a custom header</span>
<span class="sd">        @has_header X-CustomHeader</span>
<span class="sd">    &quot;&quot;&quot;</span>

    <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;has_header&#39;</span>

    <span class="k">def</span> <span class="nf">pre_process</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">response</span><span class="p">):</span>
        <span class="k">for</span> <span class="n">header</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">args</span><span class="p">:</span>
            <span class="k">if</span> <span class="n">header</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">response</span><span class="o">.</span><span class="n">headers</span><span class="p">:</span>
                <span class="k">raise</span> <span class="n">ContractFail</span><span class="p">(</span><span class="s1">&#39;X-CustomHeader not present&#39;</span><span class="p">)</span>
</pre></div>
</div>
</div>
<div class="section" id="detecting-check-runs">
<span id="detecting-contract-check-runs"></span><h2>正在检测检查运行<a class="headerlink" href="#detecting-check-runs" title="永久链接至标题">¶</a></h2>
<p>什么时候？ <code class="docutils literal notranslate"><span class="pre">scrapy</span> <span class="pre">check</span></code> 正在运行， <code class="docutils literal notranslate"><span class="pre">SCRAPY_CHECK</span></code> 环境变量设置为 <code class="docutils literal notranslate"><span class="pre">true</span></code> 字符串。你可以用 <a class="reference external" href="https://docs.python.org/3/library/os.html#os.environ" title="(在 Python v3.9)"><code class="xref py py-data docutils literal notranslate"><span class="pre">os.environ</span></code></a> 在以下情况下对蜘蛛或设置执行任何更改： <code class="docutils literal notranslate"><span class="pre">scrapy</span> <span class="pre">check</span></code> 用途：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">scrapy</span>

<span class="k">class</span> <span class="nc">ExampleSpider</span><span class="p">(</span><span class="n">scrapy</span><span class="o">.</span><span class="n">Spider</span><span class="p">):</span>
    <span class="n">name</span> <span class="o">=</span> <span class="s1">&#39;example&#39;</span>

    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;SCRAPY_CHECK&#39;</span><span class="p">):</span>
            <span class="k">pass</span>  <span class="c1"># Do some scraper adjustments when a check is running</span>
</pre></div>
</div>
</div>
</div>


           </div>
           
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
      
        <a href="practices.html" class="btn btn-neutral float-right" title="常用做法" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
      
      
        <a href="debug.html" class="btn btn-neutral float-left" title="调试spiders" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <p>
        
        &copy; 版权所有 2008–2020, Scrapy developers
      <span class="lastupdated">
        最后更新于 10月 18, 2020.
      </span>

    </p>
  </div>
    
    
    
    Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a
    
    <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a>
    
    provided by <a href="https://readthedocs.org">Read the Docs</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
  
 
<script type="text/javascript">
!function(){var analytics=window.analytics=window.analytics||[];if(!analytics.initialize)if(analytics.invoked)window.console&&console.error&&console.error("Segment snippet included twice.");else{analytics.invoked=!0;analytics.methods=["trackSubmit","trackClick","trackLink","trackForm","pageview","identify","reset","group","track","ready","alias","page","once","off","on"];analytics.factory=function(t){return function(){var e=Array.prototype.slice.call(arguments);e.unshift(t);analytics.push(e);return analytics}};for(var t=0;t<analytics.methods.length;t++){var e=analytics.methods[t];analytics[e]=analytics.factory(e)}analytics.load=function(t){var e=document.createElement("script");e.type="text/javascript";e.async=!0;e.src=("https:"===document.location.protocol?"https://":"http://")+"cdn.segment.com/analytics.js/v1/"+t+"/analytics.min.js";var n=document.getElementsByTagName("script")[0];n.parentNode.insertBefore(e,n)};analytics.SNIPPET_VERSION="3.1.0";
analytics.load("8UDQfnf3cyFSTsM4YANnW5sXmgZVILbA");
analytics.page();
}}();

analytics.ready(function () {
    ga('require', 'linker');
    ga('linker:autoLink', ['scrapinghub.com', 'crawlera.com']);
});
</script>


</body>
</html>